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Abstract 


Purpose: To develop and disseminate HIT evidence and evidence-based tools to improve 
healthcare decision making through the use of integrated data and knowledge management. 

Scope: To create a generalizable system to facilitate detection and clinician reporting of vaccine 
adverse events, in order to improve the safety of national vaccination programs. 

Methods: Electronic medical records available from all ambulatory care encounters in a large 
multi-specialty practice were used. Every patient receiving a vaccine was automatically 
identified, and for the next 30 days, their health care diagnostic codes, laboratory tests, and 
medication prescriptions were evaluated for values suggestive of an adverse event. 

Results: Restructuring at CDC and consequent delays in terms of decision making have made it 
challenging despite best efforts to move forward with discussions regarding the evaluation of 
ESP:VAERS performance in a randomized trial and comparison of ESP:VAERS performance to 
existing YAERS and Vaccine Safety Datalink data. However, Preliminary data were collected 
and analyzed and this initiative has been presented at a number of national symposia. 
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The authors of this report are responsible for its content. Statements in the report should not 
be construed as endorsement by the Agency for Healthcare Research and Quality or the El.S. 
Department of Health and Human Services of a particular drug, device, test, treatment, or 
other clinical service. 
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Final Report 


Purpose 


This research project was funded to improve the quality of vaccination programs by 
improving the quality of physician adverse vaccine event detection and reporting to the national 
Vaccine Adverse Event Reporting System (VAERS), via the following aims: 

Aim 1. Identify required data elements, and develop systems to monitor ambulatory care 
electronic medical records for adverse events following vaccine administration. 

Aim 2. Prepare, and securely submit clinician approved, electronic reports to the national 
Vaccine Adverse Event Reporting System (VAERS). 

Aim 3. Comprehensively evaluate ESP:VAERS performance in a randomized trial, and in 
comparison to existing VAERS and Vaccine Safety Datalink data. 

Aim 4. Distribute documentation and application software developed and refined in Aims 1 
and 2 that are portable to other ambulatory care settings and to other EMR systems. 


Scope 


Public and professional confidence in vaccination depends on reliable postmarketing 
surveillance systems to ensure that rare and unexpected adverse effects are rapidly identified. 

The goal of this project is to improve the quality of vaccination programs by improving the 
quality of physician adverse vaccine event detection and reporting to the national Vaccine 
Adverse Event Reporting System (VAERS). This project is serving as an extension of the 
Electronic Support for Public Health (ESP) project, an automated system using electronic health 
record (EHR) data to detect and securely report cases of certain diseases to a local public health 
authority. ESP provides a ready-made platform for automatically converting clinical, laboratory, 
prescription, and demographic data from almost any EHR system into database tables on a 
completely independent server, physically located and secured by the same logical and physical 
security as the EHR data itself. The ESP:VAERS project developed criteria and algorithms to 
identify important adverse events related to vaccinations in ambulatory care EHR data, and made 
attempts at formatting and securely sending electronic VAERS reports directly to the Centers for 
Disease Control and Prevention (CDC). 

Patient data were available from Epic System’s Certification Commission for Health 
Information Technology-certified EpicCare system at all ambulatory care encounters within 
Atrius Health, a large multispecialty group practice with over 35 facilities. Every patient 
receiving a vaccine was automatically identified, and for the next 30 days, their health care 
diagnostic codes, laboratory tests, and medication prescriptions are evaluated for values 
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suggestive of an adverse vaccine event. When a possible adverse event was detected, it was 
recorded, and the appropriate clinician was to be notified electronically. 

Clinicians in-basket messaging was designed to provide a preview a pre-populated report 
with information from the EHR about the patient, including vaccine type, lot number, and 
possible adverse effect, to inform their clinical judgment regarding whether they wish to send a 
report to VAERS. Clinicians would then have the option of adding free-text comments to pre¬ 
populated VAERS reports or to document their decision not to send a report. The CDC’s Public 
Health Information Network Messaging System (PHIN-MS) software was installed within the 
facilities so that the approved reports could be securely transferred to VAERS as electronic 
messages in an interoperable health data exchange format using Health Level 7 (HL7). 


Methods 


The goal of Aim 1: Identify required data elements, and develop systems to monitor 
ambulatory care electronic medical records for adverse events following vaccine administration, 
and Aim 2: Prepare, and securely submit clinician approved, electronic reports to the national 
Vaccine Adverse Event Reporting System (VAERS), was to construct the below flow of data in 
order to support the first two Aims: 


Figure 1. Overview of the ESP:VAERS project 



Existing and functioning ESP components are shown on the left, and Aims 1 and 2 on the 
right. ESP:VAERS flags every vaccinated patient, and prospectively accumulate that patient’s 
diagnostic codes, laboratory tests, allergy lists, vital signs, and medication prescriptions. A main 
component of Aim 1 was to Develop AE criteria to assess these parameters for new or abnormal 
values that might be suggestive of an adverse effect. A reporting protocol & corresponding 
algorithms were developed to detect potential adverse event cases using diagnostic codes, and 
methods were tested to identify prescriptions or abnormal laboratory values that might be 
suggestive of an adverse effect. These algorithms were designed to seek both expected and 
unexpected adverse effects. 
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This reporting protocol was approved by both internal & external partners. We initially 
prepared a draft document describing the elements, algorithms, interval of interest after 
vaccination, and actions for broad classes of post-vaccination events, including those to be 
reported immediately without delay (such as acute anaphylactic reaction following vaccination), 
those never to be reported (such as routine check-ups following vaccination) and those to be 
reported at the discretion and with additional information from the attending physician through a 
feedback mechanism. The draft was then widely circulated as an initial / working draft for 
comment by relevant staff in the CDC and among our clinical colleagues at Atrius. In addition to 
review by the internal CDC Brighton Collaboration liaison, this protocol has also received 
review & comment via the CDC’s Clinical Immunization Safety Assessment (CISA) Network. 

The goal of Aim 2 was the Development of HL 7 messages code for ESP: VAERS to ensure 
secure transmission to CDC via PHIN-MS. The HL7 specification describing the elements for 
an electronic message to be submitted to Constella, the consultants engaged by CDC for this 
project was implemented. Synthetic and real test data was been generated and transmitted 
between Harvard and Constella. However, real data transmissions of non-physician approved 
reports to the CDC was unable to commence, as by the end of this project, the CDC had yet to 
respond to multiple requests to partner for this activity. 

The goal of Aim 3 was to Comprehensively evaluate ESP: VAERS performance in a 
randomized trial, and in comparison to existing VAERS and Vaccine Safety Datalink data. 

We had initially planned to evaluate the system by comparing adverse event findings to those 
in the Vaccine Safety Datalink project—a collaborative effort between CDC’s Immuni z ation 
Safety Office and eight large managed care organizations. Through a randomized trial, we 
would also test the hypothesis that the combination of secure, computer-assisted, clinician- 
approved, adverse event detection, and automated electronic reporting will substantially increase 
the number, completeness, validity, and timeliness of physician-approved case reports to VAERS 
compared to the existing spontaneous reporting system; however, due to restructuring at CDC 
and consequent delays in terms of decision making, it became impossible to move forward with 
discussions regarding the evaluation of ESP: VAERS performance in a randomized trial, and 
compare ESP:VAERS performance to existing VAERS and Vaccine Safety Datalink data. 
Therefore, the components under this particular Aim were not achieved. 

Aim 4 Distribution of documentation and application software developed and refined in 
Aims 1 and 2 that are portable to other ambulatory care settings and to other EMR systems has 
been successfully completed. Functioning source code is available to share under an approved 
open source license. ESP:VAERS source code is available as part of the ESP source code 
distribution. It is licensed under the LGPL, an open source license compatible with commercial 
use. We have added the ESP:VAERS code, HL7 and other specifications and documentation to 
the existing ESP web documentation and distribution resource center http://esphealth.org, 
specifically, the Subversion repository available at: 
http://esphealth.org/trac/ESP/wiki/ESPVAERS. 
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Results 


Preliminary data were collected from June 2006 through October 2009 on 715,000 patients, 
and 1.4 million doses (of 45 different vaccines) were given to 376,452 individuals. Of these 
doses, 35,570 possible reactions (2.6 percent of vaccinations) were identified. This is an average 
of 890 possible events, an average of 1.3 events per clinician, per month. These data were 
presented at the 2009 AMIA conference. 

In addition, ESP:VAERS investigators participated on a panel to explore the perspective of 
clinicians, electronic health record (EHR) vendors, the pharmaceutical industry, and the FDA 
to wards systems that use proactive, automated adverse event reporting. 

Adverse events from drugs and vaccines are common, but underreported. Although 25% of 
ambulatory patients experience an adverse drug event, less than 0.3% of all adverse drug events 
and 1-13 % of serious events are reported to the Food and Drug Administration (FDA). 

Fikewise, fewer than 1% of vaccine adverse events are reported Fow reporting rates preclude or 
slow the identification of “problem” drugs and vaccines that endanger public health. New 
surveillance methods for drug and vacci ne adverse effects ar e needed. Barriers to reporting 
include a lack of clinician awareness, uncertainty about when and what to report, as well as the 
burdens of reporting: reporting is not part of clinicians’ usual workflow, takes time, and is 
duplicative. Proactive, spontaneous, automated adverse event reporting imbedded within EHRs 
and other information systems has the potential to speed the identification of problems with new 
drugs and more careful quantification of the risks of older drugs. 

Unfortunately, there was never an opportunity to perform syst em performance asses sments 
because the necess ary CPC contacts w ere no longer available and the CDC consultants 
responsible for receiving data were no longer responsive to our multiple requests to proceed with 
testing and evaluation. 


Inclusion of AHRQ Priority Populations 

The focus of our project was the Atrius Health (formerly HealthOne) provider & patient 
community. This community serves several AHRQ inclusion populations, specifically low- 
income and minority populations in primarily urban settings. 

Atruis currently employs approximately 700 physicians to serve 500,000 patients at more 
than 18 office sites spread throughout the greater Metropolitan Boston area. The majority of 
Atruis physicians are primary care internal medicine physicians or pediatricians but the network 
also includes physicians from every major specialty. 

The entire adult and pediatric population served by Atruis was included in our adverse event 
surveillance system (ESP:VAERS). Atruis serves a full spectrum of patients that reflects the 
broad diversity of Eastern Massachusetts. A recent analysis suggests that the population served 
by Atruis is 56% female, 16.6% African American, 4% Hispanic. The prevalence of type 2 
diabetes in the adult population is 5.7%. About a quarter of the Atruis population is under age 18. 
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List of Publications and Products 


ESP:VAERS [source code available as part of the ESP 
source code distribution]. Licensed under the GNU Lesser 
General Public License (LGPL), an open source license 
compatible with commercial use. Freely available under an 
approved open source license at: http://esphealth.org. 
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